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Abstract  -  We  present  a  summary  of  a  body  of  work 
executed  over  the  last  eight  years  addressing  battlespace 
representations  in  the  domains  of  air,  space,  and  cyber.  We 
couch  this  presentation  in  an  historical  context  and  relate  it 
to  design  principles  as  well  as  user-centered  design 
processes.  We  summarize  the  work  in  each  of  the  domains 
and  conclude  with  some  thoughts  about  supporting  real 
work  in  applied  settings. 
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1  Introduction 

Visualization  has  a  long  history.  Its  value  is  well 
illustrated  by  the  work  of  Dr.  John  Snow.  Between  1831 
and  1854,  Great  Britain  suffered  four  major  outbreaks  of 
cholera.  Throughout  this  20-year  period,  and  despite 
volumes  of  recorded  data  on  casualties,  the  situation  baffled 
the  local  authorities  as  well  as  the  scientific  community. 
Snow  created  an  overlay  of  the  victims  in  a  single 
neighborhood,  and  observed  that  the  epicenter  of  the 
epidemic  converged  at  a  water  pump  on  the  corner  of  Broad 
Street  and  Cambridge  Street,  i  The  map  can  be  viewed  at 
http://www.ph.ucla.edu/epi/snow/highressnowmap.html. 

Similarly,  battlespace  representations  have  a  long  history  of 
evolution  and  use.  One  of  the  authors  once  worked  with  a 
senior  leader  in  the  Cockpit  Integration  Office  at  Wright- 
Patterson  Air  Force  Base  (WPAFB)  who  frequently  declared 
his  NUTS  theory:  “Nothing  under  the  sun.”  The  theory  was 
based  on  the  notion  that  technology  changes,  but  the  ideas 
stay  the  same.  Thus,  3D  displays,  while  a  current 
technology  trend,  were  used  in  WWII;  they  were  cardboard 
models  of  targets  carried  in  the  bombers  to  allow 
bombardiers  to  see  how  the  target  would  look  with  current 
shadows.  See  Figure  1  for  an  example. 

Command  and  control  has  changed  as  well  as  targeting. 
Figure  2  illustrates  a  WW  II-era  command  center  built  in 
tunnels  near  Dover,  England.  The  plotting  table  was  used  to 
keep  track  of  aircraft  movements  as  radioed  in  by  spotters. 
The  concepts,  in  both  cases,  3D  models  and  command  and 
control  centers  are  thus  demonstrated  to  not  be  novel;  the 
concepts  stay  the  same,  but  the  technology  to  support  them  is 
ever-changing.  The  ever-changing  technology  landscape 
leads  developers  of  Battlespace  representations  to  both  stay 


abreast  of  current  technology  and  to  seek  invariants  that  can 
be  used  to  shape  or  guide  designs. 


Figure  1.  A  cardboard  model  used  by  bombers. 
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Figure  2.  WWII  plotting  table  in  Dover  tunnels  1. 


2  Stable  Design  Principles 

What  is  invariant  is  the  human  perceptual  and 
cognitive  apparatus  that  underlies  the  sense-making  process 
when  using  current  display  technology  and  techniques. 
Thus,  the  relationship  between  and  among  displayed  objects, 
their  color,  texture,  and  contours  and  the  use  of  those 
relationships  and  properties  in  telling  a  story, 
communicating  a  situation  to  the  user  or  operator,  is  critical. 
Display  or  graphics  designers  must  be  deeply  aware  of  the 
needs  of  their  users. 

Also  invariant  are  the  notions  of  visual  momentum,  visual 
search,  isomorphism,  vernier  acuity,  Stroop  effect,  and 
Gestalt  closure.  Graphic  or  interface  design  can  either 
induce  or  reduce  visual  search,  dependent  on  the  task  at 
hand  and  the  relationship  between  and  among  display 
elements.  The  Stroop  effect,  which  manifests  itself  as 
interference  in  the  processing  of  the  names  of  colors  written 
in  colors  other  than  those  of  the  color  being  named, 
illustrates  verbal  and  visual  conflicts  in  processing 
information. 

Strategies  such  as  Shneiderman’s  “Overview  first,  zoom 
and  filter,  then  details-on-demand.”2  or  McCracken’s 
progressive  refinement  in  which  the  broad  context  is 
conveyed  in  the  upper  left  of  the  display,  and  successively 
refined  left  to  right  and  top  to  bottom,  with  the  bridge 
between  context  and  data  being  appropriately  designed 
levels  of  abstraction/levels  of  details 

Yet  another  concept  is  the  notion  of  surface  structure  vs. 
deep  structure  of  the  interface.  This  concept  addresses 
issues  such  as  system  transport  delays,  which  can  manifest 
themselves  at  the  user  interface  and  thus  impact  the  user’s 
experience.  Ignoring  the  effect  of  time  delays  on  the 
delivery  of  data  to  dynamically  updating  displays  can  have 
deleterious,  even  catastrophic  results  (think  Ground 


Collision  Avoidance  Systems)  GCAS).  All  of  these  elements 
(and  more)  are  the  underpinning  of  good  display  design.  In 
order  to  support  “connecting  the  dots”  or  making 
information  available  “at-a-glance”  these  human  perceptual 
and  cognitive  factors  must  be  addressed.  Their 
consideration  and  integration  in  the  interface  development 
process  are  critical  for  success.  This  was  as  true  in  Snow’s 
day  as  it  is  today. 

3  Air  Force  UDOP  Family  of  Systems 

The  Air  Force  Research  Laboratory  (AFRL),  711th  Human 
Performance  Wing  (711th  HPW),  Battlespace  Visualization 
Branch  (AFRL/RHCV)  has  sponsored  a  series  of  Small 
Business  Innovative  Research  (SBIR)  efforts  dating  back  to 
2004  aimed  at  developing  Command  and  Control  (C2) 
Battlespace  Representations  for  Air,  Space,  and  Cyber.  The 
Design  Knowledge  Company  has  won  five  of  those 
contracts.  These  include  the  Satellite  Threat  Evaluation 
Environment  for  Defensive  Counterspace  (STEED), 
Local/Distributed  Operational  Collaborative  User-Centered 
Support  System  (LOCUS),  4D  NETcentric  Framework  For 
OpeRational  C2  Environments  (4D  NETFORCE),  Air 
Operations  Community  (AOC)  System  Engineering  Toolkit 
(ASET),  and  Air,  Space  and  Cyber  User-Definable 
Operational  Picture/Common  Operational  Picture  (ASC& 
U/C).  These  are  summarized  in  Figure  3.  Space  Weather 
Information  Fusion  Technology  (SWIFT),  Bridge  for 
Usable  Collaborative  Knowledge  Integration  (BUCK-I),  and 
Visualization  of  Information  for  Satellite  Tactical 
Applications  (VISTA)  are  plug-ins  to  the  core  UDOP 
technology  and  support  battlespace  operations  in  air,  space, 
and  cyber.  These  capabilities  have  matured  to  the  point  of 
being  the  work  environment  for  the  JMS  program  of  record, 
and  being  used  in  the  2011  Joint  Expeditionary  Force 
Experiment  and  Advanced  Concept  Event.  An  overview  is 
provided  in  Figure  3. 


Air  Force  UDOP  Family  of  Systems 

A' 


STEED 

o 

A 

JSARS 

JSIPUDOP 

SWIFT,  BUCK-1,  VISTA 

JMS  Program  of  Record 

JEFX  2011,  ACE  2011 

JMSO 

JEFX,  ACE  Integrated 

LOCUS  O 

4D  NETFORCE  (Cross-Domain  UDOP) 
ASET  © 

Operational  Environment, 
Ryan  Center  Install 

JEFX  2011,  ACE  2011 

J 

ASC&U/C 


Invited  UDOP  Presentation  to  ESC/HSJJO 


Figure  3.  Family  of  Agile  Work  Environments 


4  User-Centered  Design 

The  United  States  Air  Force  has  a  long  history  of  user- 
centered  design.  Each  of  the  projects  described  began  with 
an  understanding  of  the  user  population.  Cognitive  task 


analyses  were  performed,  pertinent  documents  reviewed, 
users  interviewed,  and  work  processes  observed.  The  key  to 
our  success  is  that  TDKC  cognitive  and  human  factors 
engineers  work  closely  with  our  software  development  team 
to  ensure  that  we  build  a  work  environment  that  focuses  on  a 
single,  unified  user  interface  to  multiple  systems  that  is 
tailorable  for  individual  users.  This  integrated  team 
approach  is  critical  because  our  JMS  experience  has 
demonstrated  that  legacy  tools  as  well  as  many  different 
vendors  may  be  contributing  services  to  support  diverse 
functionality,  each  having  different  styles  and  thus  requiring 
the  cognitive  and  perceptual  integration  task  to  be  done  in 
close  collaboration  with  the  UDOP  software  development. 
We  next  describe  space,  air,  and  cyber  visualizations  in  the 
form  of  Agile  Work-centered  Environments  (AWEs) 
developed  for  AFRL/RHC. 


Figure  4.  Using  the  Space  Order  of  Battle  Perspective  to 
select  a  GLOBALSTAR  asset. 

4.2  Air 


4.1  Space 

Space  assets  are  critical  to  our  military  superiority,  thus 
making  them  targets  for  attacks.  These  attacks  were  difficult 
to  detect  using  current  (2003)  tools  available  at  Space 
Operations  Squadrons  (SOPS)  or  Joint  Space  Operational 
Center  (JSpOC)  because  the  degradation  effects  may  be 
gradual  and  appear  to  be  isolated.  Improved  user  work- 
centered  technologies  were  needed  to  assist  the  operator  in 
detecting  otherwise  imperceptible  trends  and  to  rapidly 
gather  the  information  needed  to  report  the  incident.  Neural 
networks  were  being  refined  for  counterspace  operations  and 
could  detect  some  trends,  but  this  technology  lacked  the 
flexibility  needed  for  warfighters  to  perform  their  own 
analyses.  Current  (2003)  technology  lacked  advanced 
collaboration  methods  to  communicate  results  of  analyses 
across  echelons;  an  example  of  this  is  communicating  a 
finding  from  SOPS  to  the  14th  Air  Force.  To  address  this 
problem,  AFRL/RH  sought  innovative  proposals  to  develop 
a  Work-Centered  Support  System  (WCCS)  that  bridged  the 
gap  between  the  analyst  and  the  numerous  information 
sources  available,  as  well  as  collaboration  across  echelons. 
The  resulting  STEED  project  evolved  through  a  series  of 
steps  into  the  Joint  Space  Operations  Center  (JSpOC) 
Mission  System  (JMS)  UDOP.  An  example  of  one  of  the 
JMS  UDOP  perspectives  is  shown  in  Figure  4.  JMS  is  built 
on  a  net-centric,  Service-Oriented  Architecture  (SO A). 
STEED  employed  a  variety  of  visualization  approaches  to 
build  situation  awareness  support.  These  include  2D  and  3D 
geovisualizations  using  NASA  WorldWind,  standard 
graphical  user  interface  tools  such  as  pick  lists  and  tree 
structures,  task-customized  small  multiples,  and  various 
time-based  views,  among  others. 


The  4D  NETFORCE  effort  assessed  the  requirements 
to  transform  the  existing  TDKC  STEED  into  a  Command 
and  Control  (C2)  environment  to  support  work  in  Air 
Operations  Centers  (AOCs),  providing  integrated  2D  and  3D 
spatial  and  temporal  displays.  The  result  of  our 
investigations,  working  with  the  AOC  experts  from 
Teledyne-CollaborX,  was  that  this  goal  was  fully  achievable. 
We  identified  open  source  tools  to  integrate  various 
information  sources  such  as  InfoWorkSpace  (IWS),  email, 
databases  and  electronic  chat,  and  created  an  ability  to  run 
Microsoft  products  such  as  Word,  Excel,  and  PowerPoint 
inside  of  the  4D  NETFORCE  framework;  this  is  critical 
because  many  AOC  products  are  captured  using  Microsoft 
(MS)  tools.  A  sophisticated  data  loader  supports  the  import 
of  data  from  a  variety  of  data  sources,  including  legacy 
stove-piped  systems,  JSeries  messages,  and  net-centric 
services.  4D  NETFORCE  has  been  used  at  the  2011  Joint 
Expeditionary  Force  Experiment  (JEFX)  and  Advanced 
Concept  Event.  It  incorporates  all  of  the  UDOP  core 
technology  found  in  JMS.  A  sample  perspective  from  the 
4D  NETFORCE  capability  is  shown  in  Figure  5. 


Figure  5.  Air  picture  with  order  of  battle,  map,  timeline  and 
tasking  views. 


development  processes  and  usability  testing  ensure  reliable, 
usable  end  products. 


4.3  Cyber 

Major  General  Ellen  Pawlikowski,  while  Deputy 
Director  of  the  National  Reconnaissance  Office  (NRO),  was 
asked  at  the  2009  AMOS  conference  about  the  biggest  threat 
to  space  assets.  Her  answer  was  “Cyber.” 

Focusing  on  visualizations,  data  handling  and  design,  and 
operator  aiding,  the  ASC&UC  project  is  extending  the 
existing  UDOP  capability  into  a  comprehensive  command 
and  control  (C2)  visualization  space  and  shared 
collaboration  space  for  Air,  Space,  and  Cyber  warfighters. 
Meeting  this  challenge,  The  Design  Knowledge  Company 
(TDKC)  is  building  a  principled  framework  for  visualization 
to  ensure  visual  coherence  over  the  three  domains. 
Developing  new  innovative  collaboration  and  visualization 
components  that  can  integrate  across  space,  air,  and  cyber 
domains  will  yield  a  robust  and  adaptable  framework  for 
lowering  the  cost  of  current  and  future  developments  and 
provide  a  much-needed  synergy  across  development.  A 
preliminary  example  of  a  cyber  perspective  is  illustrated  in 
Figure  6. 


Offensive  and  Defensive  Ops 


Figure  7.  General  Elder’s  Conception  of  Game-Changing 
Integrated  C2. 


5  Conclusions 

Figure  8  borrows  another  graphic  from  Gen  Elder’s 
2007  NDIA  presentation.  This  graphic  shows  the  2007 
vision  of  moving  our  command  centers  from  having  a 
multitude  of  stovepiped  tools  and  data  bases  each  with  its 
own  display  to  a  set  of  services  that  can  share  data  and 
present  a  single  User  Defined  Operational  Picture.  In  2006- 
2007,  there  existed  in  the  AOC  some  88  different  stand¬ 
alone  tools.  During  a  CTA  conducted  with  JSpOC  operators, 
the  authors  identified  more  than  75  different  tools  in  use. 


Using  the  AWE  toolkit  and  accompanying  processes, 
TDKC  is  building  an  AWE  supporting  situation  awareness 
for  the  nuclear  arsenal.  The  AWE  toolkit  brings  the 
capability  to  develop  what  Lt.  General  Bob  Elder,  at  the 
time  Commander,  8th  Air  Force,  called  game-changing 
integration  across  domains,  illustrated  in  Figure  7.5 
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Figure  6.  Cyber  perspective  showing  map,  data,  and  small 
multiple  visualizations. 


4.4  Cross  Domain  Work  Environment 

TDKC  has  derived  a  toolkit  and  framework  from  the 
components  developed  across  all  of  the  projects  described 
above.  The  Agile  Work-centered  Environment  (AWE)  core 
contains  the  capability  that  is  supported  across  all  domains. 
AWE  allows  us  to  very  rapidly  integrate  domain-specific 
data  sources  to  develop  tailored  work  environments.  TDKC 
cognitive  engineering  processes  define  the  information  and 
knowledge  required  to  support  work4;  our  design  knowledge 
guides  the  structure  of  the  user  interface,  and  our  software 


Figure  8.  Evolution  from  legacy  stovepiped  systems  to 
services-driven  UDOP. 


We’ve  made  considerable  progress,  particularly  on  the 
technology  side.  SO  A  has  allowed  us  to  move  away  from 
stovepiped  tools  and  databases.  However,  this  doesn’t  make 
decision  maker’s  task  any  easier  unless  the  data  is  fused  and 
presented  in  a  way  that  facilitates  sensemaking.  Bringing  all 
this  data  together  requires  much  more  than  simply  hanging 
legacy  tools  or  new  applications  on  a  SOA.  Roth  et  al  assert 
that  “In  too  many  cases,  the  introduction  of  computer 
technology  has  placed  greater  requirements  on  human 
cognitive  activity.  ...systems  that  provide  access  to  more 
data  or  that  automate  more  processes  do  not  necessarily 
simplify  the  user’s  task  or  guarantee  improved 
performance. ”6  Somehow  must  get  from  an  overabundance 
of  data  to  decision  quality  information.  This  requires 
correlating  data  in  time  and  space,  putting  the  right  data 
together,  dealing  with  conflicting  data  as  well  as  metadata, 
and  presenting  the  data  to  the  decision  maker  in  a  way  that  is 
easily  understandable. 

Much  of  data  fusion  takes  place  in  the  grey  matter  of  the 
human.  Understanding  how  humans  fuse  data  to  reach 
conclusions  is  an  important  area  of  research.  Visualizations 
are  for  the  purpose  of  allowing  humans  to  understand 
something  -  to  get  meaning.  Designers  must  understand  the 
applied  work  space  to  know  what  meaning  is  needed;  they 
must  have  a  deep  understanding  of  the  real  work.  Issues 
include  more  than  providing  individual  pieces  of 
information,  but  providing  context  and  integrating  the 
information  in  a  meaningful  way.  We  must  also  consider 
facilitation  of  shared  understanding  for  collaboration. 

Creating  visual  representations  for  individual  applications  is 
easy;  creating  a  holistic  operator  environment  is  difficult. 
Operators  and  decision  makers  need  the  UDOP  to  be  a 
holistic  information  composition  expressed  thru  interactive 
visualization.  The  technology  solution  should  amplify  the 
human  cognitive  expertise  and  minimize  interaction 
overhead. 
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